Image-pickup apparatus and white-balance control method provided therefor

ABSTRACT

The present invention allows for selecting a predetermined chromatic color for extraction from among a plurality of chromatic colors based on the result of identification of acquired photographing information. When the value of a white extraction area of the photographing information is smaller than a first threshold value, it is determined whether the value of the area of the selected predetermined chromatic color is greater than or equal to a second threshold value. If the value of the predetermined chromatic-color area is determined to be greater than or equal to the second threshold value, white balance is controlled by bringing the value of the predetermined chromatic-color area near to a predetermined target color value.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image-pickup apparatus featuring anauto-white-balance-control technology and a white-balance control methodprovided therefor.

2. Description of the Related Art

In recent years, systems configured to automatically control whitebalance by using the output of an image-pickup element without using anexternal sensor have become the mainstream of auto-white-balance controlsystems used for a video camera. Japanese Patent Laid-Open No. 05-64219has proposed a white-balance control device having the above-describedconfiguration.

The white-balance control device that has been proposed by JapanesePatent Laid-Open No. 05-64219 obtains color-difference signals (R-Y andB-Y) and a luminance signal (Y) from color signals of red (R), green(G), and blue (B), and extracts the color-signal component correspondingto a color close to white from the luminance signal and the colorsignals. Further, the white-balance control device performs control sothat the average of the extracted color signals becomes equal to thevalue of a white color (an achromatic color) determined to be thetarget.

Further, technologies for estimating the color temperature of the lightsource by using a chromatic color such as a flesh color in the casewhere the signal components corresponding to the color close to whiteare few in number in relation to the entire screen image have beenavailable (e.g., refer to Japanese Patent Laid-Open No. 2003-333616).

Conventional technologies will be described next in detail withreference to FIG. 13.

FIG. 13 is a block diagram showing an exemplary image-pickup apparatushaving a conventional white-balance control device.

In FIG. 13, an imaging system 1301 includes a lens configured to form alight flux of the subject into an image and an image-pickup elementconfigured to perform photoelectric conversion for incident light. Aluminance-and-color-signal generation unit 1302 converts a signalgenerated by the imaging system 1301 into a luminance signal (Y) andcolor signals (R, G, and B).

A luminance processing unit 1303 performs unsharp masking and/or gammaprocessing for the luminance signal. A white-balance amplification unit1304 amplifies the color signals (R, G, and B) transmitted from theluminance-and-color-signal generation unit 1302 based on a white balancegain.

A color processing unit 1305 performs generation of color-differencesignals (R-Y and B-Y), color-balance adjustment, and so forth. An outputunit 1306 outputs an image signal. A white extraction unit 1307 extractsthe signal corresponding to a color close to white from the imagesignal.

A chromatic-color extraction unit 1308 extracts the signal correspondingto a color close to a predetermined chromatic color from the imagesignal. A white-balance gain control unit 1309 determines the whitebalance gain of the white-balance amplification unit 1304 based on theresult of the extraction performed by each of the white extraction unit1307 and the chromatic-color extraction unit 1308.

Next, operations of the above-described system will be described.

The imaging system 1301 performs photoelectric conversion for lightwhich is made incident through an optical system by using theimage-pickup element and transmits the image signal to theluminance-and-color-signal generation unit 1302. Theluminance-and-color-signal generation unit 1302 generates the luminancesignal (Y) and the color signals (R, G, and B) based on the transmittedimage signal, and transmits the luminance signal to the luminanceprocessing unit 1303 and transmits the color signals to thewhite-balance amplification unit 1304.

The luminance processing unit 1303 performs the unsharp masking, thegamma processing, and so forth for the luminance signal. On the otherhand, the white-balance amplification unit 1304 amplifies the colorsignals (R, G, and B) based on the white balance gain transmitted fromthe white-balance gain control unit 1309 which will be described laterand transmits the amplified color signal to the color processing unit1305.

The color processing unit 1305 generates the color-difference signals(R-Y and B-Y) based on the color signals (R, G, and B), performs colorprocessing such as color-balance adjustment for the color-differencesignals, and transmits the color-difference signals to the output unit1306, the white extraction unit 1307, and the chromatic-color extractionunit 1308. The output unit 1306 externally transmits the luminancesignal and the color-difference signals.

The white extraction unit 1307 extracts the signal corresponding to acolor close to a white color based on the luminance signal (Y) and thecolor-difference signals (R-Y and B-Y).

FIG. 14A shows a white extraction range 1401 of which data is extractedas a white-color area. FIG. 14A shows a color-difference (R-Y and B-Y)plane and the white extraction range 1401 is the range of a color closeto white (achromatic color).

The white extraction unit 1307 extracts a signal that is included in apredetermined luminance range and that is included in the whiteextraction range 1401. Further, the white extraction unit 1307calculates the average of the extracted color signals(extracted-white-color average) and the amount of white-color signalsobtained through white extraction (white-color amount), and transmitsinformation about the extracted-white-color average and the white-coloramount to the white-balance gain control unit 1309.

The chromatic-color extraction unit 1308 extracts the signalcorresponding to a color close to a flesh color from the luminancesignal (Y) and the color-difference signals (R-Y and B-Y).

FIG. 14B shows a flesh-color extraction range 1402 of which data isextracted as a flesh-color area.

The chromatic-color extraction unit 1308 extracts a signal that isincluded in a predetermined luminance range and that is included in aflesh-color extraction range 1402. Further, the flesh-color extractionunit 1308 calculates the average of the extracted color signals(extracted-flesh-color average) and the amount of extracted flesh-colorsignals (flesh-color amount), and transmits information about theextracted-flesh-color average and the flesh-color amount to thewhite-balance gain control unit 1309.

The white-balance gain control unit 1309 controls the white balance gainbased on the result of the color-signal extraction performed by each ofthe white extraction unit 1307 and the chromatic-color extraction unit1308.

FIG. 15 is a flowchart showing processing procedures performed by thewhite-balance gain control unit 1309 shown in FIG. 13 so as to calculatethe white balance gain.

According to FIG. 15, it is determined whether the value of thewhite-color amount is greater than or equal to a predetermined thresholdvalue at step S1501. That is to say, it is determined whether a whitearea large enough to control the white balance is included in an image.If the value of the white-color amount is greater than or equal to thethreshold value, the processing advances to step S1503. If the value ofthe white-color amount is smaller than the threshold value, theprocessing advances to step S1502.

At step S1502, it is determined whether the white-balance control isperformed based on the flesh-color signal when the image includes asmall white-color amount. More specifically, it is determined whetherthe image includes a flesh-color amount large enough to control thewhite balance. If the value of the flesh-color amount is greater than orequal to a predetermined threshold value, the processing advances tostep S1504. If the value of the flesh-color amount is smaller than thethreshold value, the processing advances to step S1505.

At step S1503, the white balance gain is calculated based on theextracted-white-color average. More specifically, a white balance gainwhich brings the extracted-white-color average near to the target white(the origin point of the color-difference plane) is calculated.

FIG. 16A shows details of the white-balance-gain control attained basedon the extracted-white-color average. FIG. 16A shows anextracted-white-color average 1601 and a point 1602, which is the originpoint of the color-difference plane.

The white-balance gain control unit 1309 determines the white balancegain of a red (R) signal and a blue (B) signal so that theextracted-white-color average 1601 is brought near to the origin point(achromatic color) 1602 of the color-difference plane, and transmitsinformation about the gain to the white-balance amplification unit 1304.

According to FIG. 16A, the white balance gain is determined so that theposition of the extracted-white-color average 1601, which is shown inFIG. 16A, is brought near to the origin point 1602. Consequently, thegain of the red (R) signal is decreased and that of the blue (B) signalis increased.

At step S1504, the white balance gain is calculated based on theextracted-flesh-color average. More specifically, a white balance gainwhich brings the extracted-flesh-color average near to the targetflesh-color value is calculated.

FIG. 16B shows details of the white-balance-gain control attained basedon the extracted-flesh-color average. FIG. 16B shows anextracted-flesh-color average 1603 and a target flesh-color value 1604.

The white-balance gain control unit 1309 determines the white balancegain of a red (R) signal and a blue (B) signal so that theextracted-flesh-color average 1603 is brought near to a predeterminedtarget flesh-color value, that is, the target flesh-color value 1604,and transmits information about the gain to the white-balanceamplification unit 1304.

According to FIG. 16B, the white balance gain is determined so that theextracted-flesh-color average 1603, which is shown in FIG. 16B, isbrought near to the target flesh-color value 1604. Consequently, thegain of the red (R) signal is decreased and that of the blue (B) signalis increased.

At step S1505, the currently used white balance gain is used, as it is,without calculating another white balance gain. The above-describedprocessing is performed when the white-color area and the flesh-colorarea that are included in the image are too small to make adetermination.

The white-balance amplification unit 1304 amplifies the color signalsbased on another white balance gain set for the photographed imagescorresponding to the next frame or later through the white-balancecontrol unit 1309.

The above-described configuration allows for attaining the white-balancecontrol through the use of information about a flesh color even thoughan image includes a small number of white-color signals, which makes itdifficult to attain the white-balance control.

According to the above-described conventional technologies, informationabout a flesh color has been used when an image (screen image) includesa little amount of information about a color close to the white color.However, the image does not necessarily include information about thewhite color and/or the flesh-color information at all times. Further, ifa subject image includes neither the information about the color closeto the white color nor the flesh-color information, it becomes difficultto adjust the white balance.

SUMMARY OF THE INVENTION

Accordingly, the present invention provides an image-pickup apparatusthat can extract a predetermined color suitable for a subject andperform the white-balance control even though an image includes a smallamount of white-color information and a white-balance-control methodprovided for the image-pickup apparatus.

An image-pickup apparatus according to an aspect of the presentinvention includes an identification unit configured to identifyacquired photographing information, a chromatic-color selection unitconfigured to select a predetermined chromatic color for extraction fromamong a plurality of chromatic colors based on a result of theidentification performed by the identification unit, the firstdetermining unit configured to determine whether the value of a whiteextraction area of the photographing information is greater than orequal to a first threshold value, the first white-balance control unitconfigured to control white balance by bringing the color of the whiteextraction area near to an achromatic color when the first determiningunit determines that the value of the white-extraction area is greaterthan or equal to the first threshold value, the second determining unitconfigured to determine whether the value of the area of the selectedpredetermined chromatic color is greater than or equal to a secondthreshold value when the first determining unit determines that thevalue of the white extraction area is smaller than the first thresholdvalue, and the second white-balance control unit configured to controlthe white balance by bringing the value of the predeterminedchromatic-color area near to a predetermined target color value when thesecond determining unit determines that the value of the predeterminedchromatic-color area is greater than or equal to the second thresholdvalue.

According to another aspect of the present invention, a white-balancecontrol method used for an image-pickup apparatus is provided, where themethod includes an identification step provided to identify acquiredphotographing information, a chromatic-color selection step provided toselect a predetermined chromatic color for extraction from among aplurality of chromatic colors based on the result of the identificationperformed through the identification step, the first determining stepprovided to determine whether the value of a white extraction area ofthe photographing information is greater than or equal to a firstthreshold value, the first white-balance control step provided tocontrol white balance by bringing the color of the white extraction areanear to an achromatic color when it is determined that the value of thewhite-extraction area is greater than or equal to the first thresholdvalue through the first determining step, the second determining stepprovided to determine whether the value of the area of the selectedpredetermined chromatic color is greater than or equal to a secondthreshold value when it is determined that the value of the whiteextraction area is smaller than the first threshold value through thefirst determining step, and the second white-balance control stepprovided to control the white balance by bringing the value of thepredetermined-chromatic-color area near to a predetermined target colorvalue when it is determined that the value of thepredetermined-chromatic-color area is greater than or equal to thesecond threshold value through the second determining step.

An image-pickup apparatus according to another aspect of the presentinvention includes a white extraction unit configured to extract asignal included in a predetermined white extraction range from acquiredphotographing information, the first white-balance control unitconfigured to control white balance by bringing a color extracted by thewhite extraction unit near to an achromatic color, a chromatic-colorextraction unit configured to extract a signal included in the range ofa predetermined chromatic color from the photographing information, thesecond white-balance control unit configured to control the whitebalance by bringing the value of a predetermined chromatic colorextracted by the chromatic-color extraction unit near to a predeterminedtarget color value, and a chromatic-color selection unit configured toselect the predetermined chromatic color for extraction from among aplurality of chromatic colors based on the acquired photographinginformation.

According to another aspect of the present invention, a white-balancecontrol method used for an image-pickup apparatus is provided, where themethod includes a white extraction step provided to extract a signalincluded in a predetermined white extraction range from acquiredphotographing information, the first white-balance control step providedto control white balance by bringing a color extracted through the whiteextraction step near to an achromatic color, a chromatic-colorextraction step provided to extract a signal included in the range of apredetermined chromatic color from the photographing information, thesecond white-balance control step provided to control the white balanceby bringing the value of a predetermined chromatic color extractedthrough the chromatic-color extraction step near to a predeterminedtarget color value, and a chromatic-color selection step provided toselect the predetermined chromatic color for extraction from among aplurality of chromatic colors based on the acquired photographinginformation.

An image-pickup apparatus according to another aspect of the presentinvention includes a white extraction unit configured to extract asignal included in a predetermined white extraction range from acquiredphotographing information, the first estimation unit configured toestimate the first color temperature based on a color signal extractedby the white extraction unit, a chromatic-color extraction unitconfigured to extract a signal included in the range of a predeterminedchromatic color from the photographing information, the secondestimation unit configured to estimate the second color temperaturebased on the color signal of a chromatic color extracted by thechromatic-color extraction unit, a calculation unit configured tocalculate the parameter of white balance based on the first and secondestimated color temperatures, and a chromatic-color selection unitconfigured to select the predetermined chromatic color for extractionfrom among a plurality of chromatic colors based on the acquiredphotographing information.

According to another aspect of the present invention, a white-balancecontrol method used for an image-pickup apparatus is provided, where thewhite-balance control method includes a white extraction step providedto extract a signal included in a predetermined white extraction rangefrom acquired photographing information, the first estimation stepprovided to estimate the first color temperature based on a color signalextracted through the white extraction step, a chromatic-colorextraction step provided to extract a signal included in the range of apredetermined chromatic color from the photographing information, thesecond estimation step provided to estimate the second color temperaturebased on a color signal of a chromatic color extracted through thechromatic-color extraction step, a calculation step provided tocalculate the parameter of white balance based on the first and secondestimated color temperatures, and a chromatic-color selection stepprovided to select the predetermined chromatic color for extraction fromamong a plurality of chromatic colors based on the acquiredphotographing information.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image-pickup apparatus according to afirst embodiment of the present invention.

FIG. 2A shows an exemplary photographed image obtained through theimage-pickup apparatus shown in FIG. 1.

FIG. 2B shows another exemplary photographed image obtained through theimage-pickup apparatus shown in FIG. 1.

FIG. 2C shows another exemplary photographed image obtained through theimage-pickup apparatus shown in FIG. 1.

FIG. 3 is a flowchart illustrating chromatic-color-for-extractiondetermining processing procedures performed by achromatic-color-extraction control unit shown in FIG. 1.

FIG. 4 shows the flesh-color area of a subject image obtained throughthe image-pickup apparatus shown in FIG. 1.

FIG. 5 shows chromatic-color extraction ranges defined by theimage-pickup apparatus shown in FIG. 1.

FIG. 6 is a flowchart showing white-balance gain calculation (control)processing procedures executed by a white-balance gain control unitshown in FIG. 1.

FIG. 7 shows target chromatic-color values determined by theimage-pickup apparatus shown in FIG. 1.

FIG. 8 is a block diagram of an image-pickup apparatus according to asecond embodiment of the present invention.

FIG. 9 shows the relationships between photographing modes and chromaticcolors for extraction that are used by the image-pickup apparatus shownin FIG. 8.

FIG. 10 is a block diagram of an image-pickup apparatus according to athird embodiment of the present invention.

FIG. 11 is a flowchart showing white-balance gain calculation (control)processing procedures executed by a white-balance gain control unitshown in FIG. 10.

FIG. 12A shows the relationship between an extracted-white-color averageand a color temperature that are specified by the image-pickup apparatusshown in FIG. 10.

FIG. 12B shows the relationship between an extracted-flesh-color averageand another color temperature that are specified by the image-pickupapparatus shown in FIG. 10.

FIG. 13 is a block diagram showing the configuration of an exemplaryconventional image-pickup apparatus.

FIG. 14A shows a white-extraction range defined by the conventionalimage-pickup apparatus shown in FIG. 13.

FIG. 14B shows a flesh-color extraction range defined by theconventional image-pickup apparatus shown in FIG. 13.

FIG. 15 is a flowchart showing conventional white-balance gaincalculation (control) processing procedures executed by a white-balancegain control unit shown in FIG. 13.

FIG. 16A shows an extracted-white-color average and a target white-colorvalue that are determined by the conventional image-pickup apparatusshown in FIG. 13.

FIG. 16B shows an extracted-chromatic-color average and a targetchromatic-color value that are determined by the conventionalimage-pickup apparatus shown in FIG. 13.

DESCRIPTION OF THE EMBODIMENTS

Various embodiments of the present invention will now be described indetail in accordance with the accompanying drawings.

An example where a predetermined color used when an image shows fewcolors close to white is selected based on information about theidentification of a subject will be described, as a first embodiment ofthe present invention.

FIG. 1 is a block diagram of an image-pickup apparatus according to thefirst embodiment.

In FIG. 1, an imaging system 101 includes a lens configured to form alight flux of the subject into an image and an image-pickup element. Aluminance-and-color-signal generation unit 102 converts a signalgenerated through the imaging system 101 into a luminance signal (Y) andcolor signals (R, G, and B).

A luminance processing unit 103 performs unsharp masking and/or gammaprocessing for the luminance signal. A white-balance amplification unit104 amplifies the color signals (R, G, and B) transmitted from theluminance-and-color-signal generation unit 102 based on a white balancegain.

A color-processing unit 105 performs generation of color-differencesignals (R-Y and B-Y), color-balance adjustment, and so forth. An outputunit 106 outputs image signals. A white extraction unit 107 extracts thesignal corresponding to a color close to white from the image signals.

A chromatic-color extraction unit 108 extracts the signal correspondingto a color close to a predetermined chromatic color from the imagesignals. A white-balance gain control unit 109 determines the whitebalance gain of the white-balance amplification unit 104 based on theresult of the extraction performed by each of the white extraction unit107 and the chromatic-color extraction unit 108.

A subject identification unit 110 identifies a photographed image of thesubject. A chromatic-color-extraction control unit 111 controls thechromatic-color extraction based on the identification result.

Next, operations of a system having the above-described configurationwill be described.

The processing procedures performed to extract the signal correspondingto the color close to white, which are started by the image-pickupsystem 101 and completed by the white extraction unit 107, are the sameas those started by the image-pickup system 1301 and completed by thewhite extraction unit 1307, as illustrated in an exemplary related artshown in FIG. 13, and therefore redundant description thereof isomitted.

Next, the subject identification unit 110 will be described. The subjectidentification unit 110 identifies what the picked-up subject image is.According to the first embodiment, the subject identification unit 110identifies whether the photographed image includes images of “person'sface”, “blue sky”, and “green trees and plants”.

Each of FIGS. 2A, 2B, and 2C shows an exemplary subject. FIG. 2A showsan image including the image of a person's face, FIG. 2B shows an imageincluding the image of the blue sky, and FIG. 2C shows an imageincluding the image of green trees and plants.

Various methods have been proposed to achieve a unit configured toidentify subjects including a person's face or the like, and any of theabove-described methods may be used for the subject identification unit.To identify the “person's face”, for example, data of the edge of theface is extracted based on a luminance signal Y of a photographed image,and the extracted edge data is matched with data of the pattern of apart of the face, where the face data had already been stored, so thatthe face is identified. Data of the result of the subject identificationis transmitted to the chromatic-color extraction control unit 111.

The chromatic-color-extraction control unit 111 selects a singlechromatic color which should be extracted by thechromatic-color-extraction control unit 111, so as to control the whitebalance.

FIG. 3 is a flowchart illustrating chromatic-color-for-extractiondetermining processing procedures performed by thechromatic-color-extraction control unit 111 shown in FIG. 1.

The chromatic-color-extraction control unit 111 determines the chromaticcolor for extraction based on the subject-identification-result datatransmitted from the subject identification unit 110.

According to FIG. 3, it is determined whether the photographed imageincludes the image of the person's face at step S301. If thephotographed image includes the face image, the processing advances tostep S304. Otherwise, the processing advances to step S302.

At step S302, it is determined whether the photographed image includesthe image of the blue sky. If the photographed image includes theblue-sky image, the processing advances to step S305. Otherwise, theprocessing advances to step S303.

At step S303, it is determined whether the photographed image includesthe image of the green trees and plants. If the photographed imageincludes the image of the green trees and plants, the processingadvances to step S306. Otherwise, the processing advances to step S307.

Steps S301 to S303 function, as an identification unit (subjectidentification unit) configured to identify what the acquiredphotographing information (subject) is.

At step S304, a flesh color is selected as the chromatic color forextraction. At step S305, a blue color is selected as the chromaticcolor for extraction. At step S306, a green color is selected as thechromatic color for extraction. At step S307, it is determined that nochromatic color is extracted. Then, the processing procedures arefinished.

Steps S304 to S306 function, as a chromatic-color selection unitconfigured to select a predetermined chromatic color for extraction fromamong a plurality of chromatic colors based on the result of theidentification performed by the identification unit.

The subject information and information about the chromatic color forextraction that are determined based on the above-described flow aretransmitted to the chromatic-color extraction unit 108 and thewhite-balance gain control unit 109.

The chromatic-color extraction unit 108 extracts a chromatic color basedon the subject-identification information and thechromatic-color-for-extraction information. When the flesh color isexemplarily selected as the chromatic color for extraction, data of acolor that falls within the range of the flesh color is extracted froman area defined in the image, the area including the face image, basedon the subject-identification information.

FIG. 4 shows an example where data of the flesh color is extracted. FIG.4 shows a photographed image of a subject including a person's face. InFIG. 4, a diagonally shaded area identified as the person's face is aface area 401. The chromatic-color extraction unit 108 extracts data ofa color included in a flesh-color extraction range shown in FIG. 12Bfrom the face area 401.

Further, the chromatic-color extraction unit 108 calculates the averageof the values of extracted flesh color and the amount of extractedflesh-color signals (flesh-color amount), and transmits informationabout the average of the extracted flesh-color values and theflesh-color amount to the white-balance gain control unit 109.

According to the above-described embodiment, thechromatic-color-extraction control unit 111 exemplarily selects theflesh color, as the chromatic color for extraction. However, if the bluecolor is selected as the chromatic color for extraction, a color signalincluded in a blue-color extraction range is extracted from an areaidentified as the blue sky. Further, if the green color is selected asthe chromatic color for extraction, a green signal is extracted from anarea identified as trees and plants.

FIG. 5 shows a blue-color-extraction range 501 and agreen-color-extraction range 502 that are exemplarily defined on acolor-difference plane. In FIG. 5, data of the average of the values ofextracted color signals and the extraction amount (the blue-color amountand/or the green-color amount) is transmitted to the white-balance gaincontrol unit 109, as is the case where the flesh color is selected asthe chromatic color for extraction.

Next, operations of the white-balance gain control unit 109 will bedescribed. The white-balance gain control unit 109 controls the whitebalance gain based on the result of the color-signal extractionperformed by each of the white-extraction unit 107 and thechromatic-color extraction unit 108.

FIG. 6 is a flowchart showing white-balance gain calculation (control)processing procedures executed by the white-balance gain control unit109 shown in FIG. 1.

According to FIG. 6, it is determined whether the value of thewhite-color amount is greater than or equal to a first predeterminedthreshold value at step S601. Namely, it is determined whether the imageincludes a white area large enough to control the white balance.

Implementation of step S601 functions as the first determining unitconfigured to determine whether the value of the white-extraction areaof the photographing information is greater than or equal to the firstpredetermined threshold value.

When the value of the white-color amount is greater than or equal to thefirst predetermined threshold value, the processing advances to stepS603. Otherwise, the processing advances to step S602.

If the image includes few white-color signals, it is determined whetherthe white-balance control can be attained based on chromatic-colorsignals. Namely, it is determined whether the value of the amount of apredetermined chromatic color (any one of the flesh color, the bluecolor, and the green color in the above-described embodiment) that isincluded in the image and that is selected by thechromatic-color-extraction control unit 111 is greater than or equal toa second predetermined threshold value.

Implementation of step S602 functions as the second determining unitconfigured to determine whether the value of the area of thepredetermined chromatic color selected by the chromatic-color selectionunit is greater than or equal to the second predetermined thresholdvalue when the first determining unit determines that the value of thewhite-extraction area is smaller than the first predetermined thresholdvalue.

When the value of the amount of the predetermined chromatic colorincluded in the image is greater than or equal to the secondpredetermined threshold value, the processing advances to step S604.When the value of the amount of the predetermined chromatic color issmaller than the second predetermined threshold value, the processingadvances to step S606. The first and second predetermined thresholdsvalues can be the same, although they are not required to be the same.

At step S603, the white balance gain is calculated based on theextracted-white-color average and information about the calculated whitebalance gain is transmitted to the white-balance amplification unit 104.Since the processing of step S603 is the same as the control clarifiedin “Description of the Related Art” with reference to FIG. 13, redundantdescription thereof is omitted.

Implementation of step S603 functions as the first white-balance controlunit configured to control the white balance by bringing the color ofthe white-extraction area near to an achromatic color when the firstdetermining unit determines that the value of the white-extraction areais greater than or equal to the first predetermined threshold value.

At step S604, data of the target color value corresponding to thepredetermined chromatic color is acquired. For example, when thechromatic-color-extraction control unit 111 determines to extract dataof the color of the blue sky, the target color value of the blue coloris acquired.

FIG. 7 shows exemplary target color values. FIG. 7 shows the targetflesh-color value 701 corresponding to a person, the target blue-colorvalue 702 corresponding to the blue color of the sky, and the targetgreen-color value 703 corresponding to the green color of the trees andplants. Since the distributions of the blue color of the sky and thegreen color of the trees and plants are wider than that of the fleshcolor of the person, the range of each of the target blue-color value702 and the target green-color value 703 is larger than that of thetarget flesh-color value 701.

At step S605, the white balance gain is calculated based on theextracted-chromatic-color average. More specifically, a white balancegain which makes the average of the values of the extracted chromaticcolor fall within the range of the target color value acquired at stepS604 is calculated. At that time, the white balance may not becontrolled so that the extracted-chromatic-color average coincides withthe center of each of the ranges of the target color values. Namely, thewhite-balance control is stopped when the average falls within the rangeof the target color value.

Implementation of step S605 functions as the second white-balancecontrol unit configured to control the white balance by bringing thevalue of the area of a predetermined chromatic color near to apredetermined target-color value when the second determining unitdetermines that the value of the predetermined chromatic-color area isgreater than or equal to the second predetermined threshold value.

At step S606, another white balance gain is not calculated, but thehitherto used white balance gain is used, as it is. The above-describedprocessing is performed when the image includes a small white area and asmall chromatic-color area, the chromatic-color area corresponding to asubject. After steps S603, S605, and S606 are performed, the processingprocedures shown in the flowchart of FIG. 6 are finished.

Information about the white balance gain calculated based on theabove-described flowchart is transmitted to the white-balanceamplification unit 104. The white-balance-amplification unit 104amplifies color signals by using another white balance gain set forphotographed images of the next frame or later.

As described above, the image-pickup apparatus of the above-describedembodiment includes the subject identification unit (the subjectidentification unit 110) and the chromatic-color selection unit (thechromatic-color-extraction control unit 111) configured to select apredetermined chromatic color used for any color other than the whitecolor based on the result of identification performed by the subjectidentification unit.

The above-described configuration allows for adjusting the white balanceeven though the photographed image of the subject includes a small whitearea by extracting data of a color suitable for the subject and bringingthe value of the extracted color data near to a target value.

According to the above-described embodiment, the chromatic-color data isextracted after the chromatic color for extraction is determined.However, the time when the chromatic color for use is determined may beafter the extraction of the chromatic-color data. For example, afterextracting data of all of the colors corresponding to ranges from whichdata of colors including the flesh color, the blue color, and so forthis extracted, which chromatic color should be used to control the whitebalance may be determined based on the subject-identificationinformation.

Further, according to the above-described embodiment, the control isperformed based on the chromatic color when the subject image includes asmall area having a color close to the white color. However, it becomespossible to perform control by using the closer to the target value ofthe white-extraction average and the chromatic-color-extraction average.For example, if the value of the flesh-color-extraction result is closeto the target flesh-color value, the white balance gain is calculatedbased on the flesh-color-extraction result, even though the subjectimage includes a sufficiently large white area.

Further, according to the above-described embodiment, the flesh color ofthe person, the blue color of the blue sky, and the green color of thetrees and plants have been described as the predetermined chromaticcolors. However, the type of the predetermined chromatic color is notlimited only to those of the above-described colors. For example, thered color of a sunset may be used as the chromatic color. In that case,the subject identification unit 110 determines whether the subject imageincludes the sunset image.

Further, when two or more subject images are detected at the same time,the chromatic colors are selected in order of decreasing precedence,that is, in the order of the flesh color of the person, the blue colorof the blue sky, and the green color of the trees and plants accordingto the processing flow shown in FIG. 3. However, the order in which thechromatic colors are selected is not limited to the above-describedprecedence order. For example, in a scene where the subject includes aperson's face, the main subject is the person as is often the case sothat the most important chromatic color becomes the flesh color.However, as for the blue color and the green color other than the fleshcolor, a priority may be given to either of them. Further, it becomespossible to perform the following control, for example. Namely, acomparison is made between the occupation of the blue sky area in theimage and that of the green-trees-and-plants area in the image, so as todetermine the area having the occupation larger than that of the other.After that, the blue color or the green color, which corresponds to thearea having the larger occupation, is selected.

Further, in the above-described embodiment, only one target value ispermanently determined for the chromatic color. However, without beinglimited to the above-described embodiment, a plurality of target valuesmay be determined. For example, the target flesh-color valuescorresponding to individual predetermined persons may be provided. Inthat case, the subject identification unit 110 identifies thepredetermined persons and selects the target flesh-color value that hadalready been related to each of the persons.

Further, a setting unit may be provided so that the user can set thepriority order of a subject by operating a menu, for example. In thatcase, if a plurality of subjects is detected from the same screen image,the color of a subject with a high priority is extracted as thechromatic color for extraction.

An example where the chromatic color for extraction is selected based onthe photographing mode used when the image was photographed will next bedescribed as a second embodiment of the present invention.

FIG. 8 is a block diagram of an image-pickup apparatus according to thesecond embodiment.

In FIG. 8, blocks designated by reference numerals 101, 102, 103, 104,105, 106, 107, 108, 109, and 111 are equivalent to the processing blocksdesignated by the same reference numerals described with respect to thefirst embodiment with reference to FIG. 1. In the second embodiment, anoperation unit 801 configured to accept data of an operation, the databeing transmitted from the user, and a photographing-mode-acquisitionunit 802 configured to acquire photographing-mode information areprovided in place of the subject identification unit 110 of the firstembodiment.

Next, operations of the above-described system will be described.

Since processing procedures performed to extract the signalcorresponding to a color close to white through from the imaging system101 to the white extraction unit 107 are the same as those described inthe first embodiment, redundant descriptions thereof are omitted.

The operation unit 801 accepts transmitted data of photographing modeset by the user. Here, the photographing mode including “portrait mode”,“fresh green mode”, and so forth is provided to specify a subject and/ora scene for photographing and perform image processing based on thecharacteristics of the subject and/or the scene.

The photographing mode can be switched to different photographing modebased on data of operations performed by the user, the data beingtransmitted from the user to the operation unit 801. Thephotographing-mode acquisition unit 802 acquires and transmits thephotographing-mode information to the chromatic-color-extraction controlunit 111.

The chromatic-color-extraction control unit 111 determines whichchromatic color should be extracted based on the photographing mode.

FIG. 9 shows the relationships between the photographing modes and thechromatic colors for extraction. The relationship between thephotographing mode and the chromatic color is determined so that data ofthe color of a subject considered as the main subject in eachphotographing mode is extracted.

For example, when the “portrait mode” is selected, photographing isperformed on the precondition that the main subject is a person. In thatcase, therefore, the flesh color is selected as the chromatic color forextraction. Similarly, the green color is selected as the chromaticcolor for extraction when the “fresh green mode” is selected, the redcolor is selected as the chromatic color for extraction when“autumnal-leave mode” is selected, the blue color is selected as thechromatic color for extraction when “blue-sky mode” is selected, and thered color is selected when “sunset mode” is selected. Information aboutthe chromatic color-for-extraction selected in the above-describedmanner is transmitted to the chromatic-color extraction unit 108.

The chromatic-color extraction unit 108 extracts data of a chromaticcolor based on the chromatic color-for-extraction information. When theflesh color is selected as the chromatic color for extraction, thechromatic-color extraction unit 108 extracts data of color signalsincluded in a flesh-color-extraction range 1402 shown in FIG. 14B fromthe image, and calculates the extracted-flesh-color average and theflesh-color amount. If a chromatic color other than the flesh color isselected, the chromatic-color extraction unit 108 also extracts data ofcolor signals, calculates the average of the extracted chromatic colorand the chromatic-color amount, and transmits information about thecalculated average and the calculated amount to the white-balance gaincontrol unit 109.

The white-balance gain control unit 109 controls the white balance gainbased on the result of the color-signal extraction performed by each ofthe white extraction unit 107 and the chromatic-color extraction unit108. Since the flow of the white-balance-gain calculation is basicallythe same as that shown in the flowchart of FIG. 6 described in the firstembodiment, the redundant detailed description thereof is omitted.However, the white-balance gain control unit 109 obtains a target colorvalue suitable for the photographing mode, so as to obtain the targetchromatic-color value at step S604 shown in FIG. 6.

Information about the white balance gain calculated by the white-balancegain control unit 109 is transmitted to the white-balance amplificationunit 104. The white-balance amplification unit 104 amplifies the colorsignal based on another white balance gain set for photographed imagesof the next frame or later, as is the case with the first embodiment.

As described above, the image-pickup apparatus of the above-describedembodiment includes a chromatic-color selection unit (thechromatic-color-extraction control unit 111) configured to select apredetermined chromatic color used for any color other than the whitecolor based on the photographing mode.

The above-described configuration allows for adjusting the white balanceby extracting data of the color of the subject in accordance with thephotographing mode and bringing the value of the color data near to thetarget value, even though the photographed image of the subject includesa small white area.

In the present embodiment, data of the color included in thepredetermined-chromatic-color-extraction range is extracted from theentire image. However, the subject identification information may beused in addition, as is the case with the first embodiment. For example,if the flesh color is selected as the chromatic color for extraction inthe “portrait mode”, the control may be performed as below. Namely, dataof the person's face is detected, and data of colors included in theflesh-color-extraction range is extracted from the area of the detectedface data.

Further, the photographing mode is not limited only to those describedabove. Any mode can be used so long as the mode is suitable for thesubject. For example, the “autumnal-leave mode” may be used asadditional mode.

An example where the white balance is controlled by estimating the colortemperature of the light source based on the chromatic-color area of thesubject image will next be described as a third embodiment of thepresent invention.

FIG. 10 is a block diagram of an image-pickup apparatus according to thethird embodiment.

The configuration of the above-described image-pickup apparatus is thesame as that of the image-pickup apparatus of the first embodimentillustrated in FIG. 1 except for the addition of acoordinate-transformation unit 1001.

Next, operations of the above-described system will be described.

Since the processing procedures that are started by the imaging system101 configured to pickup an image and that are finished by the outputunit 106 configured to externally transmit data of the image are thesame as those performed in the first embodiment, redundant descriptionthereof is omitted.

A color signal transmitted from the luminance-and-color-signalgeneration unit 102 is transmitted to the coordinate-transformation unit1001. The coordinate-transformation unit 1001 converts R, G, and B colorsignals into a luminance signal (Y) and color-difference signals (R-Yand B-Y).

The converted signals are transmitted to the individual white extractionunit 107, chromatic-color extraction unit 108, and subjectidentification unit 110. The subject identification unit 110 identifiesthe subject and the chromatic-color extraction unit 108 selects achromatic color for extraction as is the case with the first embodiment.

Further, since the processing procedures performed by the whiteextraction unit 107 and the chromatic-color extraction unit 108 are thesame as those performed in the first embodiment, redundant descriptionthereof is omitted.

The present embodiment is different from the first embodiment in thatthe color temperature of the light source is estimated based on theresult of the color extraction. The color temperature is estimatedthrough the white-balance gain control unit 109.

FIG. 11 is a flowchart showing processing procedures performed by thewhite-balance gain control unit shown in FIG. 10 to calculate (control)the white balance gain.

In FIG. 11, it is determined whether the value of the white-color amountextracted by the white-extraction unit 107 is greater than or equal to afirst predetermined threshold value at step S1101. For example, it isdetermined whether the image includes a white area large enough tocontrol the white balance. If the value of the white-color amount isgreater than or equal to the first predetermined threshold value, theprocessing advances to step S1103. Otherwise, the processing advances tostep S1102.

At step S1102, it is determined whether the value of the amount of apredetermined chromatic color (the flesh color, the blue color, thegreen color, etc.) selected by the chromatic-color-extraction controlunit 111 when the image includes a small number of white-color signalsis greater than or equal to a second predetermined threshold value, thepredetermined chromatic color being included in the image. If the valueof the amount of the predetermined chromatic color included in the imageis greater than or equal to the second predetermined threshold value,the processing advances to step S1104. Otherwise, the processingadvances to step S1105. The first and second predetermined thresholdsvalues can be the same, although they are not required to be the same.

At step S1103, the color temperature of the light source is estimatedbased on the extracted-white-color average. Implementation of step S1103functions as the first estimation unit configured to estimate the firstcolor temperature based on color signals extracted by a white extractionunit (the white extraction unit 107).

FIG. 12A shows the relationship between the extracted-white-coloraverage and the color temperature on the color-difference plane. In FIG.12A, a white-extraction range 1201 and a black-body-radiation axis 1202are shown. Namely, the white-color data moves on theblack-body-radiation axis 1202 in accordance with the variation of thecolor temperature. Each of reference numerals 1203, 1204, and 1205indicates the position of the white-color data, the position varyingfrom one color temperature to another.

The relationship between the color temperature and the white color isdetermined in advance by plotting color temperatures obtained when apiece of white paper or the like is photographed for each colortemperature. When the extracted-white-color average is at the position1203, it can be estimated that the color temperature of the light sourceis at 2000 K based on the relationship between the color temperature andthe white-color data. Similarly, when the extracted-white-color averageis at the position 1204, it can be estimated that the color temperatureof the light source is at 4000 K. Further, when theextracted-white-color average is at the position 1205, it can beestimated that the color temperature of the light source is at 6000 K.

At step S1104, the color temperature of the light source is estimatedbased on the extracted-chromatic-color average. Implementation of stepS1104 functions as the second estimation unit configured to estimate thesecond color temperature based on the signals of the chromatic color,which are extracted by a chromatic-color extraction unit (thechromatic-color extraction unit 108).

Here, an example where the chromatic-color-extraction control unit 111selects the flesh color as the chromatic color will be described.

FIG. 12B shows the relationship between the extracted-flesh-coloraverage and the color temperature on the color-difference plane. In FIG.12B, a flesh-color-extraction range 1211 and a flesh-color-change axis1212, where the flesh color changes in accordance with a change in thecolor temperature of the light source. Each of reference numerals 1213,1214, and 1215 indicates the position of the flesh-color data, theposition varying from one color temperature to another.

The relationship between the color temperature and the flesh color isdetermined in advance by plotting color temperatures obtained when theflesh color of a person is photographed. When the extracted-flesh-coloraverage is at the position 1213, for example, it can be estimated thatthe color temperature of the light source is at 2000 K based on therelationship shown in FIG. 12B. Similarly, when theextracted-flesh-color average is at the position 1214, it can beestimated that the color temperature of the light source is at 4000 K.Further, when the extracted-flesh-color average is at the position 1215,it can be estimated that the color temperature of the light source is at6000 K.

Further, when the blue color of the blue sky and/or the green color ofthe trees and plants other than the flesh color is selected, determiningthe relationship between the color temperature and the color in advanceallows for estimating the color temperature.

Step S1105 is processing performed when the white-color area and/or thechromatic-color area used to calculate the color temperature is small insize. In that case, it can be estimated that the color temperature isequivalent to a predetermined color temperature (e.g., the colortemperature of the daytime sunlight).

At step S1106, the white balance gain corresponding to the estimatedcolor temperature is calculated. Implementation of step S1106 functionsas a calculation unit configured to calculate the parameter of the whitebalance based on the first and second estimated color temperatures.

Here, the parameter of the white balance denotes the gain of the whitebalance, which amplifies a plurality of color signals included in animage signal for each channel. At that time, the value of the whitebalance gain corresponding to each of the estimated color temperaturesis obtained in advance. Then, the above-described processing proceduresare finished.

Information about the white balance gain calculated based on theabove-described flow is transmitted to the white-balance amplificationunit 104. The white-balance amplification unit 104 amplifies a colorsignal based on the set white balance gain.

As described above, the image-pickup apparatus according to the presentembodiment includes a chromatic-color selection unit (thechromatic-color-extraction control unit 111) configured to select thepredetermined chromatic color for use other than the white color basedon the result of the identification performed by the subjectidentification unit, so as to estimate the light source based on theselected chromatic color.

The configuration of the present embodiment allows for extracting dataof a color suitable for a subject and estimating the color temperatureof the light source even though the photographed image of the subjectincludes a small white area.

According to the present embodiment, either of the color temperatureestimated based on the extracted-white-color average and that estimatedbased on the extracted-chromatic-color average is used. However,performing control so that the final color temperature is calculated bycombining the results of the two color-temperature estimations with eachother constitutes another embodiment of the present invention.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2008-067873 filed on Mar. 17, 2008, which is hereby incorporated byreference herein in its entirety.

1. An image-pickup apparatus comprising: an identification unitconfigured to identify acquired photographing information; achromatic-color selection unit configured to select a predeterminedchromatic color for extraction from among a plurality of chromaticcolors based on a result of the identification performed by theidentification unit; a first determining unit configured to determinewhether a value of a white extraction area of the photographinginformation is greater than or equal to a first threshold value; a firstwhite-balance control unit configured to control white balance bybringing a color of the white extraction area near to an achromaticcolor when the first determining unit determines that the value of thewhite extraction area is greater than or equal to the first thresholdvalue; a second determining unit configured to determine whether a valueof an area of the selected predetermined chromatic color is greater thanor equal to a second threshold value when the first determining unitdetermines that the value of the white extraction area is smaller thanthe first threshold value; and a second white-balance control unitconfigured to control the white balance by bringing the value of thepredetermined chromatic-color area near to a predetermined target colorvalue when the second determining unit determines that the value of thepredetermined chromatic-color area is greater than or equal to thesecond threshold value, wherein when the identification unit identifiesa face area as the acquired photographing information, thechromatic-color selection unit selects skin color for extraction as thepredetermined chromatic color and the second determining unit determineswhether a value of an area of the skin color is greater than or equal toa second threshold value, and wherein when the identification unit doesnot identify a face but identifies a blue sky area as the acquiredphotographing information, the chromatic-color selection unit selectsblue color as the predetermined chromatic color and the seconddetermining unit determines whether a value of an area of the blue coloris greater than or equal to a second threshold value.
 2. Theimage-pickup apparatus according to claim 1, wherein the photographinginformation is subject information and the identification unit is asubject identification unit.
 3. A white-balance control method used foran image-pickup apparatus, the method comprising: an identification stepprovided to identify acquired photographing information using anidentification unit; a chromatic-color selection step provided to selecta predetermined chromatic color for extraction from among a plurality ofchromatic colors based on a result of the identification performedthrough the identification step using a chromatic-color selection unit;a first determining step provided to determine whether a value of awhite extraction area of the photographing information is greater thanor equal to a first threshold value; a first white-balance control stepprovided to control white balance by bringing a color of the whiteextraction area near to an achromatic color using a first white-balancecontrol unit when it is determined that the value of the whiteextraction area is greater than or equal to the first threshold valuethrough the first determining step; a second determining step providedto determine whether a value of an area of the selected predeterminedchromatic color is greater than or equal to a second threshold valuewhen it is determined that the value of the white extraction area issmaller than the first threshold value through the first determiningstep; and a second white-balance control step provided to control thewhite balance by bringing the value of the predetermined-chromatic colorarea near to a predetermined target color value using a secondwhite-balance control unit when it is determined that the value of thepredetermined-chromatic-color area is greater than or equal to thesecond threshold value through the second determining step, wherein whenthe identification step identifies a face area as the acquiredphotographing information, the chromatic-color selection step selectsskin color for extraction as the predetermined chromatic color and thesecond determining step determines whether a value of an area of theskin color is greater than or equal to a second threshold value, andwherein when the identification step does not identify a face butidentifies a blue sky area as the acquired photographing information,the chromatic-color selection step selects blue color as thepredetermined chromatic color and the second determining step determineswhether a value of an area of the blue color is greater than or equal toa second threshold value.
 4. The white-balance control method used forthe image-pickup apparatus according to claim 3, wherein thephotographing information is subject information and the identificationstep is a subject identification step.